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(54) Multimedia conferencing system over ISDN network 



(57) Real-time multimedia services are transmitted 
over a hybrid network including a nonguaranteed quality 
of service packet switched local area network and a cir- 
cuit switched ISDN wide area network having a central- 
ized multimedia bridge located within the wide area 
network The local area networks and multimedia 
bridge are interconnected via ISDN routers. An algo- 
rithm executed by the nrujitimedia bridge receives sig- 



nals from the packet switched network and detects the 
absence of properties needed for real-time audio visual 
services. The data signals are processed to conrpen- 
sate for the absence of the properties and then are 
transmitted over the wide area network to enable real- 
time audio visual services. 
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Description 
Technicai Reld 

This invention relates to multimecfia conferencing 
services and more particularly relates to such services 
provided over multiple nonguaranteed quality of service 
local area networks In geographically dispersed loca- 
tions interconnected by an ISDN wide area network in 
which bridging is performed in the ISDN network. 

Background of the Invention 

Real-time multimedia services have been provided 
in the past by connecting signals over circuit switched 
networks having guaranteed quality of service. 
Although this approach results in reasonable quality, it is 
cost prohibitive for many potential users. Most offices 
desiring multimedia services, such as video conferenc* 
ing. are served by a packet switched local area network 
(LAN) providing nonguaranteed quality of service. Such 
networks are inherently unreliable for multimedia. The 
multimedia packet streams coming from such networks 
do not have the proper sequencing. The arrival time 
between the packets may have significant variations 
known as delay jitter, and some packets may be lost 
Moreover, the recovered bit streams of audio and video 
do not maintain required tenporal relationships to 
insure both intramedia and intermedia synchronization 
so that lip-synchronization is maintained. Since the cost 
of installing a guaranteed quality of service network in 
such offices is prohibitive, the adoption of multimedia 
services in most offices has been severely restricted. 

One attempt to provkJe video and audio services is 
described in U.S. Patent No. 4.866,704 (Bergman, filed 
March 16. 1988). Bergman describes the conversion of 
data to T1 frames and then an additional conversion to 
packets for distrikxjtion on a wide band ftk^r optic ring. 
This arrangement is contrary to the existing local area 
network structure found in most offices, and therefore 
would required the installation of new networks at pro- 
hibitive expense. 

Another approach to transmitting data from one 
location having a local area network to a second loca- 
tion having a local area network via a T1 network is 
described in U.S. Patent No. 5.384.835 (Wheeler et al.. 
filed January 12. 1993). Wheeler et al. teach such a net- 
work arrangement for distrttxiting text data and still 
graphic data. However, they do not describe any 
method or apparatus for provkJing real time video and 
audio over the local area and T1 networks. 

Summaty of the Invention 

My invention enables real-time multimedia confer- 
encing services to be provided to dispersed locations 
interconneted by a wide area network from multimedia 
signals originating at the locations. The multimedia sig- 



nals are audio, vkleo or data signals, or any combina- 
tion of such signals. The locations employ 
nonguaranteed quality of service packet switched local 
area networks which degrade one or more properties 

5 required by the multimedia signals. In such an environ- 
ment, multimedia conferencing signals n)ay k>e provkied 
by receiving the multimedia signals from the local area 
networks at the wide area network, bridging the multi- 
media signals in the wide area network and transmitting 

10 the bridged multimedia signals over the wide area net- 
work to the various locations. By using this method, the 
bandwidth required to transmit the multimedia signals 
between the dispersed locations is reduced. 

According to another aspect of my invention, the 

15 at^ence of the required properties is detected in the 
wide area network. The multimedia signals then are 
processed to compensate for the absence of the prop- 
erties. The processed multimedia signals are transmit- 
ted to the dispersed locations over the wkJe area 

20 network in order to improve the rQyltimedia services. 

My invention also may be enhanced by detecting 
the at^ence of the properties in the multimedia signals 
at the dispersed locations and processing the multime- 
dia signals to compensate for the absent properties. 

26 

Brief Description of the Drawings 

In the drawing. 

30 Ffgure 1 illustrates a preferred form of end-to-end 
network configurations for multipoint multimedia 
conferencing services through interconnection of 
nonguaranteed quality of service (NQQOS) local 
area networks (LANs) to an integrated services dig- 

35 ital network (ISDN) via ISDN routers in accordance 
with my invention; 

Rgure 2 depicts a preferred form of high-level end- 
to-end protocol architecture for the multipoint multi- 
media conferencing services through interconnec- 
40 tion of the nonguaranteed quality of service 
(NGQOS) local area networks (LANs) to the inte- 
grated services digital network (ISDN) via ISDN 
routers in accordance with my invention; 
Rgure 3 shows a preferred form of nonguaranteed 
45 quality of service (NGQOS) local area networks 
(LANs) based multimedia workstation^rsonal 
computer (MMWS/PC) protocol architecture in 
accordance with my inverrtion; 
Rgure 4 shows a preferred form of switched non- 
50 guaranteed quality of service (NGQOS) local area 
network (LAN) hUb based multipoint multimedia 
bridging (MMB) protocol architecture in accordance 
with my invention; and 

Rgures 5 through 8 show flew charts of a preferred 
55 form of algorithm for intra and inter-media synchro- 
nization to inprovement performance of the real- 
time audio and vkjeo traffic in accordance with my 
invention. 
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E>etaned Description 

Figure 1 illustrates multimedia work stations or per- 
sonal computers 1-1, 1-2, 1-3. 1-4 and 1-5 that are 
equipped with multimedia conferencing application pro- s 
grams leased on the H.323 standard of the International 
Telecommunications Union (ITU). Computers 1-1 and 
1 -2 are connected to a shared nonguaranteed quality of 
service local area network 2-1 at a customer premises 
or location LI . Similarly, computers 1 -4 and 1 -5 are con- io 
nected to a shared nonguaranteed quality of service 
local area network 2-2 located at another customer 
premises or location l_2 which may be displaced from 
location LI by many miles. Networks 2-1 and 2-2 are 
equipped with a gatekeeper fur>ction that controls the is 
total loading on the network as described in ITU Rec. 
H.323. A conventional ISDN router 3-1 is located at 
location LI for connecting signals from network 2-1 to a 
converrtional drcuit switched narrow band ISDN wide 
area network (WAN) 5 over a link 4-1 . Signals are proc- 20 
essed in time division multiplexing fashion over the 
ISDN links 4-1 through 4-4 before being transmitted 
over network 5. Similarly, a conventional router 3-4 at 
location l_2 is connected to network 5 over a link 4-4. 
ISDN routers 3-1 and 3-4 preferably have a gatekeeper 25 
functionality. 

Still referring to Figure 1. a single multimedia work- 
station or personal computer 1-3 is connected to a 
switched local area network hub 5-2 at another cus- 
toms premises or location L3 which may be displaced 30 
from locations LI and L2 by many miles. Such an 
an^angement has superior performance when com- 
pared with networks 2-1 and 2-3. especially for multime- 
dia traffic. 

Networks 2-1 and 2-2, as well as hub 5-2. can be an 35 
Ethernet (IEEE 802.3). token ring (IEEE 802.5), fast 
Ethernet (IEEE 802.10). or FDD! (operating in the non- 
guaranteed quality of service nrxxje). 

Still referring to Rgure 1 . a multimedia bridge 6 is 
connected to a decOcated high speed nonguaranteed 40 
quality of service local area netwoi1< hub 5-1 that pro- 
vides protocol transparency with networks 2-1 , 2-2 and 
hub 5-2. Hub 5-1 is connected to multimedia bridge 6 to 
provide a guarantee of performance parameters, such 
as delay jitter, packet loss, error rate and delay. In a 45 
multipoint telephone call, a bad source may effect the 
entire conference. Bridge 6 detects the bad source and 
communicates with end computers 1-1. 1-2, 1-3. 1-4 
and 1-5. Bridge 6 also takes specific actions agreed 
upon at the time of call setup. The communication mes- so 
sages between the computers on customer premises 
LI • L3 via ISDN wide area network 5 insures quality of 
service for multimedia conferencing. 

ISDN routers 3-1 , 3-2. 3-3 and 3-4 are connected to 
local area network 2-1 , hut>s 5-1 and 5-2 and local area ss 
network 2-2. respectively. The multimedia conferencing 
applications running on computers 1-1 through 1-5 are 
made in accordance with the ITU Rec. H.323 standard. 



ISDN routers 3-1 through 3-4 are connected to network 
5 via ISDN links 4-1 through 4-4. respectively 

Multimedia bridge 6 Is equipped with a multipoint 
control unit (MCU) functionality in accordance with the 
ITU Rec. H.323 standaid, and has a general conference 
control and multipoint communication service function 
operating in accordance with the ITU Rec. T120 stand- 
ard, as well as a nnultipoint controller and multiproces- 
sor function operating in accordance with the ITU Rec 
H.323 standard. Bridge 6 also has a gatekeeper func- 
tion, such as address translation, admission conti^ol. 
bandwidth control, call control signaling, call autiioriza- 
tion, bandwidth management, and call management as 
specified in the ITU Rec H.323 standard. The communi- 
cation between conrputers 1-1 through 1-5 and the 
txidge 6 takes place in a master-slave relationship as 
described in ITU Rec H.323. Bridge 6 plays tiie role of 
the master in controlling participating computers 1-1 
through 1-5. 

Bridge 6 depacketizes ail encapsulated packets of 
the multimedia bit streams of signals that originate from 
computers 1-1 through 1-5 and provides bridging for 
audio, video, and/or data. Thus, in this specification and 
claims, multimedia signals means audio, video or data 
signals, or any combination of such signals. A point-to- 
point call initially can be processed by bridge 6 if tiie end 
stations (i.e., computers 1-1 through 1-5) want certain 
services, such as directory services, admission control, 
or address translation at the time of call setup, while tiie 
actual bearer channel is established via an optin^l 
path. 

After the packets received from computers 1-1 
through 1 -5 have been depacketized, bridge 6 executes 
an algorithm (Rgures 5-8) which serializes the packets, 
produces dummy packets for lost packets if appropriate, 
compensates for delay jitter where necessary and 
insures real-time audio and video synchronization as 
long as the end stations operate within certain perform- 
ance guidelines. Brkjge € then performs audio, video 
and data bridging in accordance with predefined 
schemes that are agreed upon at the time of call setup. 
TTie bridged audio, video and data bit streams again are 
sent to the destination end stations via ISDN routers. 
The ISDN routers perform IP multicasting in sending tiie 
multimedia traffic if necessary. As an alternative to IP 
multicasting, the ITU Rec T.I 25 multipoint communica- 
tion service (MCS) also can be used. 

The protocol architecture for end-to-end communi- 
cations for multimedia conferencing involving computer 
1-1 and multimedia bridge 6 is explained in connection 
with Figure 2. The protocol stack 13-1 running on com- 
puter 1-1 has tiie entities GCC, MCS, H.225.0yRTP. 
TCP. UDR IP. Logical link control (LLC), and medium 
access control (MAC) between the media stream 
(audio, video, and/or data) and a physical layer. Simi- 
lariy, protocol stack 13-2 of multimedia bridge 6 has ttie 
entities GCC. MCS. H.225:0/RTR TCP. UDP, IP, LLC. 
and MAC between the media stream (audio, video 
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and/or data) and a physical layer. ISDN routers 3-1 and 
3-2 communicate between computer 1-1 and bridge 6 
using internet protocol (IP) (14-1. 14-2) via nonguaran- 
teed quality of service local area network 2-1 and net- 
work hub 5-1. Network 2-1 and hub 5-1 have protocol 5 
layers LLC and MAC (1 7-1 . 1 7-2). ISDN routers 3-1 and 
3-2 encapsulate the IP over the point-to-point protocol 
(PPP or the poirrt-tOi30int multipoint (MP)) to transfer 
audio, video an-or data over ISDN links 4-1 and 4-2 to 
communicate over the ISDN wide area network 5. 10 
Router 3-1 is connected to a circuit switching ISDN 
node of network 5. 

The detailed architecture of corrputers 1 -1 through 
1 -5 is shown in Figure 3. The communication between 
the upper layer applications 100 (e.g., audio, video, is 
data, system control, and/or other services ) and OCC 
106, MCS 107. or transport interface 101 can take place 
directly as needed. However, GCC 106 communicates 
with the transport interface via MCS 107 only in accord- 
ance with the ITU Rec. T.I 20 standard. Trarvsport inter- 20 
face 101 makes the upper layer entities transparent 
from the details of the lower layers. Transport interface 

101 can be WinSock Forum's WINSOCK 2. Multimedia 
Communication Forum's (MMCF) TSl, or other inter- 
faces. 25 

The upper layer applications 100 may contain audio 
100-1. video 100-2, data 100-3. and/or other services 
100-4. Audio 105 and video 103 coming from upper 
layer applications 100 or lower layer entity H.225.0/RTP 
108 are synchronized into an entity known as synchro- so 
nization 104. Both intramedia and intermedia synchro- 
nization services are performed by entity 104. The 
reference master time dock to provide synchronization 
is received from ISDN WAN 5 based bridge 6. The LAN 
based audio and video maintains synchronization by 3S 
receiving clock signaling from the guaranteed quality of 
sen^ice ISDN WAN based bridge 6. No performance 
degradation occurs for any media stream when rt is 
transferred over ISDN WAN 5. A synchronization algo- 
rithm in accordance with this invention (Rgures 5-8) 40 
maintains synchronization for different media. Comput- 
ers 1-1 through 1-5 are free to use their own synchroni- 
zation methods. However, bridge 6 uses the 
synchronization algorithm described in connection with 
Figures 5-8. 45 

Audio 105 and video 103 bit streams coming out of 
the upper layer applications are packetized in accord- 
ance with the ITU Rec. H.225.0 protocol in entity 108. 
H,225.0 is very similar to the lETPs RTP protocol. 
There can be almost direct mapping between H.225.0 so 
and RTR The implementation details of this mapping 
function are not a part of this invention. The audio arxj 
video packets encapsulated in H.225.0 are transferred 
using lETF's UDP protocol in entity 109. Similarly, data 

102 arKl system control 106 traffic are encapsulated ss 
over the lETPs TCP protocol in erttity 110. The system 
control traffic entity 106 includes control, call control, 
and RAS (Registration, Administration, arxj Status) as 



specified in the ITU Rec. H.245 and H.225.0 protocols. 
However. RAS traffic is transferred using the UDP pro- 
tocol. Both UDP and TCP packets again are transferred 
using lEFTs IP protocol in entity 111. The IP packet is 
then transferred over the l-AN (e-g.. LAN 2-1) using the 
LAN protocol LLC in entity 112 and LAN protocol MAC 
in entity 113. 

If the packets come from the LAN to the conputer 
(e.g.. LAN 2-1 to computer 1-1), a similar de-encapsula- 
tion of packets takes place from MAC in entity 1 13 to 
LLC in entity 1 12. from LLC in entity 1 12"to IP in entity 
111. from IP in entity 1 1 1 to UDP in entity 109 or to TCP 
in entity 110. and from UDP in entity 109 to the H.225.0 
protocol in entity 108. Audio 105 and video 103 bit 
streams are recovered from the H.225.0 protocol in 
entity 108. The intermedia and intramedia synchroniza- 
tions are performed for the recovered bit streams in 
entity 104, The synchronization services in the upper 
layer (above the transport interface 101) can also be 
provided, but this service is not part of this invention. 

The protocol architecture for bridge 6 is depicted in 
Figure 4. This architecture is similar to what has been 
shown in Figure 3. Brkjge 6 has the MCU and gate- 
keeper (GK) functionalities. The MCU functions of 
bridge 6 include bridging for audio, video and/or data 
similar to the functions stated for multipoint controller 
(MC) and multipoint processor (MP) of the ITU Rec. 
H.323 standard. The GK functions of bridge 6 include 
address translation, admission control. t>andwidth con- 
trol, call control signaling, call authorization, bandwidth 
management, call management, directory services, and 
others. However, bridge 6 works as the master for com- 
puters 1-1 through 1-5 connected to the LANs in the 
customer premises (as shown in Figure 1) from system 
controlling point of view. Bridge 6 is connected to ISDN 
router 3-2 via a dedicated LAN hub (as shown in Rgure 
1) and provides a guarantee for all performance param- 
eters. The packets coming to bridge 6 from the LANs 2- 
1 and 2-2 are de-encapsulated from MAC in entity 213 
to LLC in entity 212 to IP in entity 21 1 to UDP in entity 
210 or TCP in entity 209 as explained earlier. Both intra- 
media and intermedia synchronization for audio 207 
and video 205 streams recovered from the H.225.0 
packets in entity 208 are performed in an entity known 
as synchronization 206. A synchronization algorithm 
described in Figures 5 through 8 inproves performance 
specifically targeted for the real-time traffic coming out 
of the LANs 2-1 and 2-2 and hub 5-2. Data, system con- 
trol, and other bit streams de-encapsulated from the 
TCP packets in entity 209 are sent to the upper layer 
services. 

Audio 207 and video 205 bit streams then are trans- 
ferred to the higher layer services to perform media 
brkjging for audio 200-1. video 200-2. and data 200-3 
bridging similar to specifications defined in the ITU Rec. 
H.323 standard. Every conferee sets up the commurd- 
cation for muttipoirrt multimedia conferencing via bridge 
6. A point-to-point communication flow is set up 
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between bridge 6 and each end station or end system 
participating In the conference (e.g.. conputer 1-1 and 
1 -4). Audio bridging and video brdging are performed in 
entities 200-1 and 200-2 in accordance to the criteria 
agreed upon by the participating parties. For example, 
bridge 6 can provide either video switching or video mix- 
ing. Video switching is the process of selecting the 
video that bridge 6 outputs to computers 1-1 through 1- 
5 from one source to another. The criteria used to make 
the switch nnay be determined through detection of 
change in speaker (sensed by the assodated audio 
level) or though control according to the H.245 stand- 
ard. Video mixing is the process of formatting more tiian 
one video source into a video stream that bridge 6 out- 
puts to computers 1-1 through 1-5. The details of the 
mixing criteria by the bridge 6 is not a part of tiiis inven- 
tion. Bridge 6 can prepare N audio outputs from M audio 
Inputs by switching, mixing or a combination of these. 
Audio mixing requires decoding the input audio to linear 
signals (pulse coded nrK»dulation or analog), performing 
a linear combination of the signals, and recording the 
result to the appropriate audio format. The details of 
such audio mixing also are not a part of this invention. 
Bridge 6 processes data in accordance witii the ITU 
Rec. T120 standard, and has functions like GCC in 
entity 201 and MCS in entity 202 as shown in Rgure 4. 

Figures 5 through 8 describe a preferred form of 
algorithm to provide intra- and inter-media synchroniza- 
tion to improve performance for the real-time audio arxJ 
vkieo traffic over the networks shown in Figure 1 . The 
algorithm detects the absence of properties needed for 
multimedia conferencing, such as whether appropriate 
audio and video packets have been received, whether 
such packets are synchronized, and whether a com- 
plete video frame is ready. The algorithm is executed by 
synchronization entity 104 of bridge 6 (Figure 3). The 
process begins at step 500, while the initialization is 
done in step 501 . The variat^les shown in step 501 rep- 
resent the following: 



i.j.k = 


dummy variat^es representing an integer 




number 


M = 


Total number of audio packets in a given 




audio segment 


N = 


Total number of video packets In a given 




video frame 


F = 


Total number of video frames per second 


B» 


Total number of bits per video frame 


S = 


Total number of audio samples per sec- 




ond 


G = 


Total numt>er of bits per audio sanrple 


T = 


Playout time 


Ih = 


Reference clock time 


Aeb(k) = 


Total number of bits in audio segment k 


Vto(k) = 


Total number of bits in video frame k 


4^T = 


A fixed increment in playout time 


Vth = 


A fixed tiireshold value for the inter-arrival 


time of the video packet 



Atj, = A fixed threshold value for the inter-arrival 

time of the audio packet 
Audio packet of i-th time interval 
Video packet of i-th time interval 
Inter-anrival time of the j-th video packet 
Inter-arrival time of the i-th aucGo packet 
Average inter-arrival time for the video 
packet 

Average inter-arrival time for the audio 
packet 

Initial system time 
Rrial system time 
New playout time 

The measurement of inter-arrival time for audio and 
video packets for a given playout time starts at the same 
time in step 501. The average inter-arrival times for 
video and audio packets for each source are estimated 
as shown in steps 503 and 502, respectively: 

N 

j-1 

M 
i o 1 



In step 504, the average inter-arrival time of the 
video packet for given playout time is connpared to that 
of the threshold value of the video packet inter-arrival 
time. If Va is greater than \/^, bridge 6 notifies the 
source as indicated in step 506. Ofc>taining the notifica- 
tion from bridge 6. the source checks whether the 
shared LANs, switches. ISDN routers, or the equipment 
are degrading the performance. The source tries to rec- 
tify the prot>lems or accepts the degraded mode of oper- 
ation. In step 505. audio packet inter-arrival time also is 
compared and actios similar to those described for 
steps 504 and 506 also are taken as stated in step 507 
if Aa is greater than Ath- 

In step 508. the new playout time P is estimated by 
conparing the values of T and Va- The maximum, of the 
two values is chosen as new playout time P. In step 509, 
system initial time Sj is set to refererx;e clock time Lj that 
is being maintained by ISDN WAN 5. System final time 
Sf is set by incrementing the system initial time by the 
playout interval: S, = S j + P . 

In step 510, the total bits for audio segment A^{k) 
and video frame ^/fti^) within the given playout time are 
estimated: Agb(k)=SGP and Vft,(k) = FB. It is 
expected tfiat tiie playout time will be very dose to one 
video frame time. Audio arKJ video are then processed 
simultaneously through a series of steps described in 
Figures 6 and 7 as indicated in steps 511 and 512, 
respectively. 

In step 601 (Rgure 6), the algorithm examines 



Ap(') = 
Vp(i) = 

Ci = 
Va = 



Aa = 

10 

Si = 
Sf = 
P = 



75 



20 



25 



30 



35 



40 



45 



50 
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whether all audio packets. [A p(i), i = 1 , M ] are received 
within system final time, Sf. If all packets are not 
received within the scheduled playout time, dummy 
packets are created to replace the unreceived packets 
(step 602). The creation of the dummy packets mini- 5 
mizes errors as far as practicat>le. Audio segment, 
Asb(k) tiien is formed (step 607). 

In step 603. the algorithm examines whether any 
audio packets received in step 601 do not belong to 
audio segment. As5(k). The packets that do not belong 10 
to this audio segment are pre-buffered (step 605) if the 
packets could be used in the next segment. In step 607, 
audio segment Aet>(k) is created only with those packets 
that belong to that segment In step 61 0, the audio proc- 
ess, after creation of the segment, tbilows a series of is 
steps as irxiicated in Figure 8. 

Similarly, in step 701 of Figure 7. the algorithm 
examines whether all video packets, [Vp (j). j = 1. N] 
are received within system final time. St- ff all packets 
are not received within the scheduled playout time, sys- 20 
tern final time Sf is incremented by a small fraction of 
playout time (step 702) ^T to set the new system final 
time: 

On the other hand, if all video packets are received 
within system final time Sf , the algorithm examines 
(step 703) whether any video packets not belonging to 30 
video frame Vto(k) are received. If no excess packets 
are received, the video process is sent to step 805 of 
Rgure 8 to follow a series of steps as indicated in step 
71 2. However, if there are excess video packets in video 
frame Vfb(k), all excess video packets are dropped (step 35 
706). arKl the process is sent to step 802 of Figure 8 as 
indicated in step 709. 

In step 704. the algorithm again examines whether 
all video packets. [Vp Q), j = 1. N] are received within 
new system final time, Sf. tf there are excess video 40 
packets in. video frame, Vft,(k) the excess video packets 
are dropped (step 705) and the process is then sent to 
step 802 of Figure 8 as shown in step 709. If all video 
packets are not received within the new system final 
time, the process is sent to step 707. 45 

In step 707, the algorithm examines whether 
dummy video packets can be created to fill up the gaps 
of video frame. Vfb(l^. with minimum errors. If it is possi- 
ble to create video franrie Vft,(k) using dummy packets, 
the process is then sent to step 801 of Figure 8 (step so 
713). However, all video packets [Vp 0). j = 1. N] of 
video frame Vf5(k) are dropped (step 708), if this frame 
cannot be created due to non-availability of the video 
packets, and the process is sent to step 802 of Figure 8 
as shown in step 709. ss 

In step 800 of Figure 8. the audio process (from 
step 61 0 of Rgure 6) joins to the vkieo process in step 
806. 812 or 810 depending on the video events for inter- 



media synchronization between the audio segment and 
video frame for playout. storage, or processing as 
needed by the multimedia applications. 

In step 810, a video frame (from step 712 of Rgure 
7) and an audio segment (from step 61 0 or Figure 6) are 
synchronized for playout. storage, or processing. In step 
814, the algorithm examines whether the complete 
video frame containing the necessary video packets 
along with the synchronized audio segment is ready. If 
the video frame is ready, the process is completed (step 
815): otherwise the process is repeated by going t>ack 
to step 514 of Figure 5 as indicated in step 81 1 . 

Step 802 shows that the video process received 
from step 708 goes to step 812. where audio segment. 
A5b(k) is adjusted to new system time Sf to which a new 
video playout time has fc>een set. In step 807. the audio 
segment is kept as a continuous reference medium for 
playout. storage, or processing although the video 
frame or packets have been dropped. 

In step 806, intermedia syochronization between 
the audio (from step 610 of Rgure 6) and video process 
(either from step 713 of Rgure 7 or from step 807) are 
performed for playout. storage or processing, and the 
processes are then sent to step 808. 

In step 808, the synchronization to the new playout 
time is initialized, since the video frame playout time has 
been changed from its original value. In step 813. the 
algorithm examines whether the complete video frame 
containing the necessary video packets along with the 
synchronized audio segment Is ready. If the video frame 
is ready, the process is completed (step 815); otherwise 
tiie process goes to step 809 where both audio and 
video processes then are reinitialized, going back to 
step 513 (Figure 5). 

After step 815, the processed multimedia signals 
comprising a audio segment and video frame are trans- 
mitted to tiie upper layer applications. In the case of 
multimedia bridge 6 (Rgure 1). the audio, video and 
data signals are bridged in entities 200-1 . 200-2. arKj 
200-3, respectively. The bridged multimedia signals are 
then processed through entities Oke 203. 210. 209. 21 1 . 
213. and 214. Then the processed signals are transmit- 
ted through LAN hub 5-1. router 3-2 and ISDN WAN 5 
as previously described. A receiving LAN, such as 2-2. 
processes the audio segment and video frame signals 
as previously described in connection with Figure 3. If 
necessary, synchronization entity 104 may execute the 
algorithm shown in Figures 5-8 in order to improve per- 
formance. 

In accordance with one aspect of my invention, dif- 
ferent categories of packet mode multimedia conferenc- 
ing services can be provided: premium service, semi- 
premium service, arxj non-premium service. 

Referring to Rgure 1 , in premium service, all con- 
ference participating end stations within a customer 
premises are considered to be connected to the 
switched nonguaranteed quality of service LANs where 
only the switched LAN hubs (e.g.. hub 5-2) are shared. 
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The switched LAN hubs are directfy connected to an 
ISDN WAN router (e.g., ISDN router 3-3). The traffic 
and the performance characteristics within the ISDN 
routers will satisfy the desired requirements specified in 
the premium service criteria. The ISDN WAN t>ased s 
service provider will guarantee to rrairrtain the superior 
quality of service of the packet mode multimedia confer- 
encing services, if the shared switched hubs and ISDN 
routers operate within the given performance guide- 
lines. 10 

In semi-prenrdum service mode, there can be a mix 
of switched and shared nonguaranteed quality of serv- 
ice LANs where the participating end stations are con- 
nected. In this nrKxJe. there can be a conditional 
guarantee of quality of service by the ISDN WAN t>ased is 
packet mode multimedia conferencing service provider. 

In non-premium mode, no guarantee of the quality 
of service is provided by the ISDN WAN service pro- 
vider. The service provider uses its best effort to provide 
the packet mode multimedia conferencing services. so 

Those skilled in the art recognize that the preferred 
embodiments may be altered and amended without 
departing from the true spirit arxJ scope as defined in 
the appended claims. 

25 

Claims 

1 . In a system for providing real-time multimedia con- 
ferencing services to a dispersed plurality of loca- 
tions interconnected by a wide area network from so 
multimedia signals originating at said locations hav- 
ing properties adapted for said services, said loca- 
tions employing nonguaranteed quality of service 
packet switched local area networks which degrade 
at least one of said properties, said method conv 35 
prising in combination the steps of: 

receiving said multimedia signals from said 
local area networks at said wide area network: 
bridging said multimedia signals in said wide 40 
area network; arKl 

transmitting said bridged multimedia signals 
over said wide area network to said plurality of 
locations, whereby the bandwidth required to 
transmit said multimedia signals between said 4S 
plurality of locations is reduced. 

2- A method, as claimed in daim 1 , and further conv 
prising the steps of: 

so 

detecting the absence of at least said one prop- 
erty in said multimedia signals in said wide 
area network; 

processing said multimedia signals to compen- 
sate for tiie absence of at least said one prop- ss 
erty; and 

transmitting saki processed multimedia signals 
to said kx:ations over said wide area network. 



whereljy the quality of said services at said plu- 
rality of locations is improved. 

3. A method, as claimed in daim 2, wherein said step 
of receiving comprises the step of receiving sakj 
multimedia signals by said wide area network and 
wherein said wide area network comprises an 
ISDN network. 

4. A method, as claimed in daim 3, wherein said step 
of transmitting comprises the step of time division 
multiplexing said processed multimedia signals for 
transmission on saki ISDN network. 

5. A method, as claimed in daim 2, wherein said step 
of detecting comprises the steps of: 

estimating a first time period by which a first 
group of said multimedia signals is expected to 
be received; ^ 

determining whether any of sakJ multimedia 
signals witiiin said first group is absent at the 
end of said first time period; 
increasing said first time period to a second 
time period in the event any of said multimedia 
signals is absent; and 

determinir^g whetiier any of sakJ multimedia 
signals within said group is absent at the end of - 
said second time period. 

6. A method, as claimed in daim 5, wherein said step 
of detecting further comprises the step of determin- 
ing whether any of said multimedia signals received 
within said first time period belong to a second 
group different from said first group. 

7. A method, as claimed in daim 6, wherein said st^ 
of processing comprises the step of deleting at 
least some of said multimedia signals belonging to 
said second groupi 

8. A method, as claimed in daim 6, wherein said step 
of processing comprises the step of storing at leaist 
some of said multimedia signals belonging to sakJ 
second group. 

9- A method, as claimed in daim 5. wherein said step 
of processing comprises the step of aeating one or 
more dummy groups of multimedia signals for said 
first group in the event that multimedia signals 
within said f irst group are absent. 

10. A method, as claimed in daim 5, wherein said step 
of processing further comprises the step of syn- 
chronizing said multimedia signals. 

11. A method, as claimed in daim 5, wherein sakj mul- 
timedia signals comprise video signals arKl audio 
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signals and wherein said step of processing com- 
prises the step of synchronizing said video signals 
and said audio signals. 

12. A method, as claimed in daim 5, wherein said mul- 
timedia signals comprise video signals and audio 
signals and wherein said step of detecting com- 
prises the steps of: 

estimating a first time period by which a first 
segment of said audio signals is expected to be 
received; 

determining whether any of said audio signals 
within said first segment is absent at the end of 
said first time period; 

increasing said first time period to a second 
time period in the event any of said audio sig- 
nals is absent; 

determining whether any of said audio signals 
within said segment is absent at the end of said 
second time period; 

estimating a third time period by which a first 
frame of said video signals is expected to be 
received; 

determining whether any of said video signals 
within said first frame is absent at the end of 
said third time period; 

increasing said third time period to a fourth 
time period in the event any of said video sig- 
nals is absent; and 

determining whether any of said video signals 
within said frame is absent at the end of said 
fourth time period. 

13. A method, as claimed in claim 12, wherein said step 
of detecting further comprises the steps of: 

determining whether any of said audio signals 
received within said first time period belong to a 
second segment different from said first seg- 
ment; 

determining whether any of said video signals 
received within said third time period belong to 
a second frame different from said first frame. 

14. A method, as claimed in claim 13, wherein said step 
of processing comprises the steps of: 

storing any of said audio signals belonging to 
said second segment: and 
storing any of said video signals belonging to 
said second frame. 

15. A method, as claimed in claim 13, wherein said step 
of processing comprises the step of deleting at 
least some of said video signals belonging to said 
second frame. 



receiving said processed multimedia signals at 
one of said locations over said wide area net- 
work; 

detecting the absence of at least said one prop- 
erty in said multimedia signals at said one loca- 
tion; 

processing said one location multimedia sig- 
nals to compensate for the absence of at least 
said one property; and 

using said processed one location multimedia 
signals to aeate audio visual services at said 
one location. « 

18. A method, as claimed In claim 17. wherein the step 
of detecting the absence of at least said one prop- 
erty in said one location multimedia signals com- 
25 prises the steps of: 

estimating a frst time period by which a first 
group of said one location multimedia signals is 
expected to be received; 
30 determining whether any of said one location 

multimedia signals within said first group is 
at»sent at the end of said first time period; 
increasing said first time period to a second 
time period in the event any of said one location 
35 multimedia signals is absent; and 

determining whether any of said one location 
multimedia signals within said group is absent 
at the end of said second time period. 

40 1 9. A method, as claimed in claim 1 8, wherein said step 
of detecting the absence of at least said one prop- 
erty In said received one location multimedia sig- 
nals further comprises the step of determining 
whether any of said one location multimedia signals 

45 received within said first time period belong to a 
second group different from said first group. 

20. A method, as claimed in claim 1 9. wherein said step 
of processing said one location multimedia signals 

so comprises the step of deleting at least some of said 
one location multimedia signals belonging to said 
second group. 

21 . A method, as claimed in claim 1 9. wherein said step 
55 of processing said one location multimedia signals 

comprises the step of storing at least some of said 
one location multimedia signals belonging to said 
second group. 



55 



1 6. A method, as claimed m claim 1 3, wherein said step 
of processing further comprises the step of syn- 
chronizing said audio signals within said first seg- 
ment v^'th said video signals within said first frame. 

17. A method, as claimed in claim 2, and further com- 
prising the steps of: 
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22. A method, as claimed in daim 1 8, wherein said step 
of processing said one location multimedia signals 
comprises the step of creating one or more dummy 
groups of multimedia signals for said first group of 
one location multimedia signals in the event that 5 
multimedia signals within said first group of one 
location multimedia signals is at>sent 

23. A method, as claimed in claim 1 8, wherein said step 

of processing said one location multimedia signals 10 
further connprises the step of synchronizing said 
one location multimedia signals. 

24. A method, as claimed in claim 1 , wherein said loca- 
tions employ switched local area network hubs and is 
further comprising the steps of: 

sharing only said switched local area network 
hubs; and 

operating said switched local area network 20 ^ 
hubs within predetermined performance guide- 
lines, whereby superior multimedia conferenc- 
ing services are provided with guaranteed 
quality of service parameters. 

2S 

25. A method, as claimed in claim 1 . wherein said loca- 
tions employ a mix of shared and switched local 
area networks, whereby Improved multimedia con- 
ferencing services are provided with conditional 
guarantee of quality of service parameters. 30 
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